#latency reduction

'ACE' upgrades LLMs by incrementally editing a persistent context playbook rather than fine-tuning; it yields notable accuracy gains and 80–90% reductions in adaptation latency on agent and finance benchmarks.

'Amin is an optimistic, adaptive scheduler that uses lower-bound estimates and ordered eviction to cut LLM inference latency dramatically, often matching hindsight-optimal performance and outperforming conservative algorithms by up to 5x.'

PARSCALE introduces a parallel computation approach to scale language models efficiently, reducing memory use and latency while improving performance across various tasks.